Software Development
Stream Processing with Faust
Faust: Getting Started with Stream Processing
it_pyspwfdj_01_enus
Faust: Performing Operations & Maintaining State Using Tables
it_pyspwfdj_03_enus
Faust: Stream Processing Using Models, Agents, & Channels
it_pyspwfdj_02_enus
Faust: Stream Processing Using Windowing Operations
it_pyspwfdj_04_enus
Faust: Getting Started with Stream Processing
Lesson Objectives
Faust: Getting Started with Stream Processing
- discover the key concepts covered in this course
- recall the characteristics of batch data and batch processing
- recall the characteristics of streaming data and stream processing
- list the components that make up the architecture of a stream processing system
- summarize the kinds of transformations that can be performed on streaming data
- recall the important characteristics of Faust stream processing applications
- recall the important characteristics of the Apache Kafka message delivery service
- install and set up Kafka and Faust on a local machine
- use producers and consumers to send and receive messages
- implement a simple streaming application using Faust
- run and test a Faust streaming application
- use the 'faust' command to run workers and send messages to agents
- summarize the key concepts covered in this course
Overview/Description
Faust is a stream processing library that allows you to write code to process data available as streams. In this course, you'll explore the basics of stream processing and how it fundamentally differs from batch processing. You'll start by examining the components of a stream processing system architecture, specifically the role of stream transport. You'll then investigate the Faust stream processing library, which uses native Python code for transformations on streaming data. Moving on, you'll explore what's meant by producers, consumers, and topics in Apache Kafka. You'll install Faust on your local machine as well as the Apache Kafka messaging service. You'll then use these to write a stream processing application. When you're finished with this course, you'll be able to clearly articulate the characteristics of stream processing and work with Apache Kafka and Faust to perform simple operations on input streams.
Faust is a stream processing library that allows you to write code to process data available as streams. In this course, you'll explore the basics of stream processing and how it fundamentally differs from batch processing. You'll start by examining the components of a stream processing system architecture, specifically the role of stream transport. You'll then investigate the Faust stream processing library, which uses native Python code for transformations on streaming data. Moving on, you'll explore what's meant by producers, consumers, and topics in Apache Kafka. You'll install Faust on your local machine as well as the Apache Kafka messaging service. You'll then use these to write a stream processing application. When you're finished with this course, you'll be able to clearly articulate the characteristics of stream processing and work with Apache Kafka and Faust to perform simple operations on input streams.
Target
Prerequisites: none
Faust: Getting Started with Stream Processing
Lesson Objectives
Faust: Getting Started with Stream Processing
- discover the key concepts covered in this course
- recall the characteristics of batch data and batch processing
- recall the characteristics of streaming data and stream processing
- list the components that make up the architecture of a stream processing system
- summarize the kinds of transformations that can be performed on streaming data
- recall the important characteristics of Faust stream processing applications
- recall the important characteristics of the Apache Kafka message delivery service
- install and set up Kafka and Faust on a local machine
- use producers and consumers to send and receive messages
- implement a simple streaming application using Faust
- run and test a Faust streaming application
- use the 'faust' command to run workers and send messages to agents
- summarize the key concepts covered in this course
Overview/Description
Faust is a stream processing library that allows you to write code to process data available as streams. In this course, you'll explore the basics of stream processing and how it fundamentally differs from batch processing. You'll start by examining the components of a stream processing system architecture, specifically the role of stream transport. You'll then investigate the Faust stream processing library, which uses native Python code for transformations on streaming data. Moving on, you'll explore what's meant by producers, consumers, and topics in Apache Kafka. You'll install Faust on your local machine as well as the Apache Kafka messaging service. You'll then use these to write a stream processing application. When you're finished with this course, you'll be able to clearly articulate the characteristics of stream processing and work with Apache Kafka and Faust to perform simple operations on input streams.
Faust is a stream processing library that allows you to write code to process data available as streams. In this course, you'll explore the basics of stream processing and how it fundamentally differs from batch processing. You'll start by examining the components of a stream processing system architecture, specifically the role of stream transport. You'll then investigate the Faust stream processing library, which uses native Python code for transformations on streaming data. Moving on, you'll explore what's meant by producers, consumers, and topics in Apache Kafka. You'll install Faust on your local machine as well as the Apache Kafka messaging service. You'll then use these to write a stream processing application. When you're finished with this course, you'll be able to clearly articulate the characteristics of stream processing and work with Apache Kafka and Faust to perform simple operations on input streams.
Target
Prerequisites: none
Faust: Performing Operations & Maintaining State Using Tables
Lesson Objectives
Faust: Performing Operations & Maintaining State Using Tables
- discover the key concepts covered in this course
- perform group-by operations on streams
- perform group-by and items operations on streams
- access raw events and buffer events before performing operations
- enumerate entities in a stream with a single agent and with multiple agents
- forward messages to destination topics
- filter stream records
- perform word count using tables
- save table state to an embedded RocksDB database
- perform grouping operations and understand table sharding
- compute aggregations on streaming data
- perform multiple grouping operations with the right agent structure
- process streaming elements using multiple worker instances
- recall the different kinds of sinks that can be used with a Faust agent
- summarize the key concepts covered in this course
Overview/Description
Faust streams support a wide range of operations. In this course, you'll learn how to perform several of these. You'll also work with Faust tables - which store state in the form of key-value pairs and allow for the recovery of failed processing, making Faust fault-tolerant. You'll start off by using the group by operation to designate a key used to repartition an input stream and create a new topic in Kafka. You'll then use the items() operation to access the key and message value and take() operation to buffer multiple elements in a stream. Next, you'll work with tables to conduct stateful stream processing, illustrating how table data is stored in an embedded RocksDB database. When you've finished this course, you'll be able to apply a wide range of operations on input streams and perform stateful stream processing using tables.
Faust streams support a wide range of operations. In this course, you'll learn how to perform several of these. You'll also work with Faust tables - which store state in the form of key-value pairs and allow for the recovery of failed processing, making Faust fault-tolerant. You'll start off by using the group by operation to designate a key used to repartition an input stream and create a new topic in Kafka. You'll then use the items() operation to access the key and message value and take() operation to buffer multiple elements in a stream. Next, you'll work with tables to conduct stateful stream processing, illustrating how table data is stored in an embedded RocksDB database. When you've finished this course, you'll be able to apply a wide range of operations on input streams and perform stateful stream processing using tables.
Target
Prerequisites: none
Faust: Performing Operations & Maintaining State Using Tables
Lesson Objectives
Faust: Performing Operations & Maintaining State Using Tables
- discover the key concepts covered in this course
- perform group-by operations on streams
- perform group-by and items operations on streams
- access raw events and buffer events before performing operations
- enumerate entities in a stream with a single agent and with multiple agents
- forward messages to destination topics
- filter stream records
- perform word count using tables
- save table state to an embedded RocksDB database
- perform grouping operations and understand table sharding
- compute aggregations on streaming data
- perform multiple grouping operations with the right agent structure
- process streaming elements using multiple worker instances
- recall the different kinds of sinks that can be used with a Faust agent
- summarize the key concepts covered in this course
Overview/Description
Faust streams support a wide range of operations. In this course, you'll learn how to perform several of these. You'll also work with Faust tables - which store state in the form of key-value pairs and allow for the recovery of failed processing, making Faust fault-tolerant. You'll start off by using the group by operation to designate a key used to repartition an input stream and create a new topic in Kafka. You'll then use the items() operation to access the key and message value and take() operation to buffer multiple elements in a stream. Next, you'll work with tables to conduct stateful stream processing, illustrating how table data is stored in an embedded RocksDB database. When you've finished this course, you'll be able to apply a wide range of operations on input streams and perform stateful stream processing using tables.
Faust streams support a wide range of operations. In this course, you'll learn how to perform several of these. You'll also work with Faust tables - which store state in the form of key-value pairs and allow for the recovery of failed processing, making Faust fault-tolerant. You'll start off by using the group by operation to designate a key used to repartition an input stream and create a new topic in Kafka. You'll then use the items() operation to access the key and message value and take() operation to buffer multiple elements in a stream. Next, you'll work with tables to conduct stateful stream processing, illustrating how table data is stored in an embedded RocksDB database. When you've finished this course, you'll be able to apply a wide range of operations on input streams and perform stateful stream processing using tables.
Target
Prerequisites: none
Faust: Stream Processing Using Models, Agents, & Channels
Lesson Objectives
Faust: Stream Processing Using Models, Agents, & Channels
- discover the key concepts covered in this course
- use models to represent stream elements
- create models with multiple fields and different data types
- publish messages to a Kafka topic using the pykafka library
- invoke the ask() method to await processing results from an agent
- invoke the cast() method to await processing results from an agent
- use multiple agents for chained processing of data
- use multiple agents to process elements from different topics
- use channels to send and receive messages
- create streams manually using the .stream() method
- define stream processors to transform stream elements
- summarize the key concepts covered in this course
Overview/Description
Stream processing in Faust uses native Python code, meaning you don't have to learn a new domain-specific language to perform data transformations. All you need to know is how existing concepts, such as models, work within a Faust context. Faust models allow you to specify fields and their data types and use this well-defined data structure to access streaming data. In this course, you'll learn how to represent the individual elements of a stream using Faust models. You'll work with agents, which are at the heart of every Faust stream processing application. You'll perform operations using agents and invoke agents synchronously and asynchronously from within your application. You'll then work with channels in Faust. At the end of this course, you'll have the confidence to use models, agents, and channels in the right way to build a successful application.
Stream processing in Faust uses native Python code, meaning you don't have to learn a new domain-specific language to perform data transformations. All you need to know is how existing concepts, such as models, work within a Faust context. Faust models allow you to specify fields and their data types and use this well-defined data structure to access streaming data. In this course, you'll learn how to represent the individual elements of a stream using Faust models. You'll work with agents, which are at the heart of every Faust stream processing application. You'll perform operations using agents and invoke agents synchronously and asynchronously from within your application. You'll then work with channels in Faust. At the end of this course, you'll have the confidence to use models, agents, and channels in the right way to build a successful application.
Target
Prerequisites: none
Faust: Stream Processing Using Models, Agents, & Channels
Lesson Objectives
Faust: Stream Processing Using Models, Agents, & Channels
- discover the key concepts covered in this course
- use models to represent stream elements
- create models with multiple fields and different data types
- publish messages to a Kafka topic using the pykafka library
- invoke the ask() method to await processing results from an agent
- invoke the cast() method to await processing results from an agent
- use multiple agents for chained processing of data
- use multiple agents to process elements from different topics
- use channels to send and receive messages
- create streams manually using the .stream() method
- define stream processors to transform stream elements
- summarize the key concepts covered in this course
Overview/Description
Stream processing in Faust uses native Python code, meaning you don't have to learn a new domain-specific language to perform data transformations. All you need to know is how existing concepts, such as models, work within a Faust context. Faust models allow you to specify fields and their data types and use this well-defined data structure to access streaming data. In this course, you'll learn how to represent the individual elements of a stream using Faust models. You'll work with agents, which are at the heart of every Faust stream processing application. You'll perform operations using agents and invoke agents synchronously and asynchronously from within your application. You'll then work with channels in Faust. At the end of this course, you'll have the confidence to use models, agents, and channels in the right way to build a successful application.
Stream processing in Faust uses native Python code, meaning you don't have to learn a new domain-specific language to perform data transformations. All you need to know is how existing concepts, such as models, work within a Faust context. Faust models allow you to specify fields and their data types and use this well-defined data structure to access streaming data. In this course, you'll learn how to represent the individual elements of a stream using Faust models. You'll work with agents, which are at the heart of every Faust stream processing application. You'll perform operations using agents and invoke agents synchronously and asynchronously from within your application. You'll then work with channels in Faust. At the end of this course, you'll have the confidence to use models, agents, and channels in the right way to build a successful application.
Target
Prerequisites: none
Faust: Stream Processing Using Windowing Operations
Lesson Objectives
Faust: Stream Processing Using Windowing Operations
- discover the key concepts covered in this course
- summarize how windowing operations work on input streams
- recall the different types of windows supported by Faust and their characteristics
- recall the differences between event time, ingestion time, and processing time
- implement processing time tumbling windows
- aggregate data on a per-key, per-window basis
- contrast tumbling windows and hopping windows
- use the key index to iterate over keys, values, and items in windowed tables
- implement event time hopping windows
- use web views to monitor metrics associated with workers
- handle GET, PUT, POST, DELETE, and HTTP requests with web views
- access tables from web views
- summarize the key concepts covered in this course
Overview/Description
When working with data, windows are a handy tool to accumulate data subsets from input streams and perform aggregation operations on this specific data. In this course, you'll learn how to perform stream processing through windowing operations in Faust. You'll start by examining the different windowing operations possible on input streams, including tumbling, sliding, count, session, and global windows. Next, you'll distinguish the three notions of time associated with streaming events: event, ingestion, and processing time. You'll then use Faust window features to perform windowing operations on input streams and emit aggregation results for every window. Finally, you'll use the REST API server, which all Faust applications have, to make streaming code metrics and table data accessible to the user. Once you're done with this course, you'll be able to use windowing operations via Faust and expose metrics using web views.
When working with data, windows are a handy tool to accumulate data subsets from input streams and perform aggregation operations on this specific data. In this course, you'll learn how to perform stream processing through windowing operations in Faust. You'll start by examining the different windowing operations possible on input streams, including tumbling, sliding, count, session, and global windows. Next, you'll distinguish the three notions of time associated with streaming events: event, ingestion, and processing time. You'll then use Faust window features to perform windowing operations on input streams and emit aggregation results for every window. Finally, you'll use the REST API server, which all Faust applications have, to make streaming code metrics and table data accessible to the user. Once you're done with this course, you'll be able to use windowing operations via Faust and expose metrics using web views.
Target
Prerequisites: none
Faust: Stream Processing Using Windowing Operations
Lesson Objectives
Faust: Stream Processing Using Windowing Operations
- discover the key concepts covered in this course
- summarize how windowing operations work on input streams
- recall the different types of windows supported by Faust and their characteristics
- recall the differences between event time, ingestion time, and processing time
- implement processing time tumbling windows
- aggregate data on a per-key, per-window basis
- contrast tumbling windows and hopping windows
- use the key index to iterate over keys, values, and items in windowed tables
- implement event time hopping windows
- use web views to monitor metrics associated with workers
- handle GET, PUT, POST, DELETE, and HTTP requests with web views
- access tables from web views
- summarize the key concepts covered in this course
Overview/Description
When working with data, windows are a handy tool to accumulate data subsets from input streams and perform aggregation operations on this specific data. In this course, you'll learn how to perform stream processing through windowing operations in Faust. You'll start by examining the different windowing operations possible on input streams, including tumbling, sliding, count, session, and global windows. Next, you'll distinguish the three notions of time associated with streaming events: event, ingestion, and processing time. You'll then use Faust window features to perform windowing operations on input streams and emit aggregation results for every window. Finally, you'll use the REST API server, which all Faust applications have, to make streaming code metrics and table data accessible to the user. Once you're done with this course, you'll be able to use windowing operations via Faust and expose metrics using web views.
When working with data, windows are a handy tool to accumulate data subsets from input streams and perform aggregation operations on this specific data. In this course, you'll learn how to perform stream processing through windowing operations in Faust. You'll start by examining the different windowing operations possible on input streams, including tumbling, sliding, count, session, and global windows. Next, you'll distinguish the three notions of time associated with streaming events: event, ingestion, and processing time. You'll then use Faust window features to perform windowing operations on input streams and emit aggregation results for every window. Finally, you'll use the REST API server, which all Faust applications have, to make streaming code metrics and table data accessible to the user. Once you're done with this course, you'll be able to use windowing operations via Faust and expose metrics using web views.
Target
Prerequisites: none
